Wide Compression: Tensor Ring Nets
نویسندگان
چکیده
Deep neural networks have demonstrated state-of-theart performance in a variety of real-world applications. In order to obtain performance gains, these networks have grown larger and deeper, containing millions or even billions of parameters and over a thousand layers. The tradeoff is that these large architectures require an enormous amount of memory, storage, and computation, thus limiting their usability. Inspired by the recent tensor ring factorization, we introduce Tensor Ring Networks (TR-Nets), which significantly compress both the fully connected layers and the convolutional layers of deep neural networks. Our results show that our TR-Nets approach is able to compress LeNet-5 by 11× without losing accuracy, and can compress the state-of-the-art Wide ResNet by 243× with only 2.3% degradation in Cifar10 image classification. Overall, this compression scheme shows promise in scientific computing and deep learning, especially for emerging resourceconstrained devices such as smartphones, wearables, and IoT devices.
منابع مشابه
Efficient Handling of Phase-Type Distributions in Generalized Stochastic Petri Nets
We study the introduction of transitions with Phase-type distribution ring time in (bounded) generalized stochastic Petri nets. Such transitions produce large increases of both space and time complexity for the computation of the steady state probabilities of the underlying Markov chain. We propose a new approach to limit this phenomenon while keeping full stochastic semantics of previous works...
متن کاملBT-Nets: Simplifying Deep Neural Networks via Block Term Decomposition
Recently, deep neural networks (DNNs) have been regarded as the state-of-the-art classification methods in a wide range of applications, especially in image classification. Despite the success, the huge number of parameters blocks its deployment to situations with light computing resources. Researchers resort to the redundancy in the weights of DNNs and attempt to find how fewer parameters can ...
متن کاملWOVEN FRAMES IN TENSOR PRODUCT OF HILBERT SPACES
The tensor product is the fundemental ingredient for extending one-dimensional techniques of filtering and compression in signal preprocessing to higher dimensions. Woven frames play a crucial role in signal preprocessing and distributed data processing. Motivated by these facts, we have investigated the tensor product of woven frames and presented some of their properties. Besides...
متن کاملUsing Rubber Pad Forming Process for Local Compression in Assembling of Vehicle Structure Tubes
Nowadays, tube bonding in vehicles structure is noticeably considered by many researches. In this paper, local tube compression process by a rubber ring has been studied. Firstly, simulation of the process was performed by FEM. Then, experimental test has been carried out to validate the numerical results. Numerical and experimental results were compared showing a good agreement. In the second ...
متن کاملUsing Rubber Pad Forming Process for Local Compression in Assembling of Vehicle Structure Tubes
Nowadays, tube bonding in vehicles structure is noticeably considered by many researches. In this paper, local tube compression process by a rubber ring has been studied. Firstly, simulation of the process was performed by FEM. Then, experimental test has been carried out to validate the numerical results. Numerical and experimental results were compared showing a good agreement. In the second ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- CoRR
دوره abs/1802.09052 شماره
صفحات -
تاریخ انتشار 2018